Is N-Best Dead?

We developed a faster search algorithm that avoids the use of the N-Best paradigm until after more powerful knowledge sources have been used. We found, however, that there was little or no decrease in word errors. We then showed that the use of the N-Best paradigm is still essential for the use of still more powerful knowledge sources, and for several other purposes that are outlined in the paper. 1. I N T R O D U C T I O N The N-Best Paradigm [1] was introduced originally as a means for integrating the speech recognition and language understanding components of a spoken language system. Since then, we have generalized its use for integrating into the recognition search other expensive knowledge sources (such as higher-order n-gram language models, betweenword co-articulation models, and segmental models) without increasing the search space [2]. The basic idea is that we use inexpensive knowledge sources to find N alternative sentence hypotheses. Then we rescore each of these hypotheses with the more expensive and more accurate knowledge sources in order to determine the most likely utterance. The N-Best Paradigm specifically, and multi-pass search algofithms in general, are now used widely by the speech recognition research community. Besides its use as an efficient search strategy, the N-Best Paradigm has been used extensively in several other ways [2]. Its simplicity has made it ideal as a means for cooperation between research sites. For example, we regularly send the N-Best lists of alternatives to research sites that do not have an advanced speech recognition capability (e.g., Paramax and NYU) in order that they can apply their own linguistic components for understanding or for research into alternative language modeling techniques. Another related use of the N-Best lists is for evaluation of alternative knowledge sources. New knowledge sources can be evaluated without having to integrate them into the search strategy. For example, we can determine whether a new prosodic module or linguistic knowledge source reduces the error rate when used to reorder the N-Best list. This is particularly important for knowledge sources that are not easily formulated in a left-to-ddght incremental manner. Finally, we have presented techniques for optimizing the weights for different knowledge sources, and for discriminative training [2]. In this paper we attempt to determine whether the N-Best Paradigm results in substantial search errors. If it does, then its use for the other purposes mentioned above would also be questionable. First we describe briefly how we used the N-Best paradigm in previous versions of BYBLOS. Then, we descfibe our attempts to avoid the errors that might be a result of using the N-Best paradigm. Finally, we argue that there will always be cases where the N-Best paradigm will make it possible to use some knowledge sources that would likely never be used otherwise. 2. 3 P A S S N B E S T S E A R C H S T R A T E G Y The BYBLOS system has been described previously (e.g., [3]). We reiterate here the use of the N-Best Paradigm in that system. The decoder used a 3-pass search strategy. The strategy used a forward pass followed by a backward Word-Dependent NBest search algorithm [4] using a bigram language model, within-word triphone models, and top-1 (discrete VQ) densities. The N-Best hypotheses were then rescored using crossword triphone context models, top-5 mixture densities, and trigram language model. Typically, the backward Word-Dependent N-Best pass requires about half the time required by the forward pass. Rescoring each alternative sentence hypothesis individually with cross-word triphone models only requires about 0.2 seconds per hypothesis. And rescofing the text of the hypotheses with a high-order n-gram language model [5] requires essentially no time. 3. A D M I S S I B I L I T Y It has often been asserted that the N-Best paradigm is inadmissible because when the initial N-Best list is created using weaker knowledge sources, then the answer that would have had the highest score using the stronger knowledge sources might not be within the list of alternatives, and therefore never have a chance to be rescored. This would be especially likely when the error rate is high and the utterances